The technology disclosed relates to real-time collection and flexible reporting of test data. In particular, it is useful when collecting packet counts during tests of network devices that simulate thousands or even millions of data sessions conducted through a device under test (“DUT”).
The Internet and backbone networks handle a very large volume of data traffic. The routers and switches that handle these volumes of data are much different from the equipment that ordinary people have in their homes. While the names are the same, the capacities and protocols are much different.
In addition to backbone infrastructure, so called triple play infrastructure is being deployed at a local level. Triple play infrastructure involves high volumes of data, because it delivers voice, data and HDTV services into homes, typically using a single physical channel. Both scheduled, broadcast HDTV and HDTV on-demand have the potential of creating packet loads that far exceed the data demands of even the most ambitious websurfer.
During development and prior to deployment, it is useful to test high capacity routers, switches and hybrid devices. Testing is particularly important as new features are added and old features are enhanced. Many alternative protocols exist for carrying packets from one point to another. The more alternative protocols a device attempts to accommodate, the greater the likelihood of unpredicted interactions.
Regression testing is one tool for making sure that new features do not break existing components. In regression testing, previously run-tests are repeated to assure that new functions have been implemented without impairing old ones. Elaborate scripts are often developed for regression testing.
During development and prior to deployment, conformance, functional, performance and passive testing may be conducted. Conformance testing allows developers to verify that the operation of their device conforms to established, industry-accepted standards, conventions, or rules, such as RFCs or draft standards. During functional testing, developers verify that the device does everything that it has been designed to do, including protocol support, filtering, and management functions. Performance testing provides information about levels of performance, such as throughput, frame loss, latency, route pairing, or scalability. This approach includes stress testing to show how a device behaves under load conditions. In addition, passive testing can be performed to analyze protocol performance, either intrusively or non-intrusively.
A test device that stresses a core router, for instance, may include a dozen or more interfaces that operate at 10 or 40 gigabits. In test configurations for backbone core routers or interconnect switch fabrics, multiple test devices may be combined into a single test system that generates an enormous amount of test data.
It is difficult or impossible, using conventional data processing approaches, to respond to data queries in real time, during a test. At the end of a test, it can take hours to collect test sample data and marshal it into a database.
An opportunity arises to introduce a new data collection and distribution architecture that is adapted to test devices. Better, more timely and more responsive testing may result, with significant benefits to those who develop and deploy high-volume routers and switches.